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(57) A method, a computer readable medium, and 
a system are provided for controlling computer system 
operation during execution of the computer system. At 
least one attribute change of a computer system to oc- 
cur during the execution of the computer system is iden- 
tified. The attribute change is associated with a state 
change in the computer system such that indication of 
the state change triggers the attribute change. An index 
point is identified at which point the state change is to 
occur The state change is associated with the index 
point, and when the index point is reached during the 
course of computer system execution, the attribute 
change is triggered automatically upon the index point 
being reached. The method, computer readable medi- 
um, and system can be used for controlling computer 
system execution in a broadcast environment or used 
to emulate computer system execution. 
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^■flted patent applications entitled 'METHOD AND 
SYSTEM FOR AUTOMATIC CONTROL OF GRAPHf 

fo„=: T?Ill "^^""3 '^^^ number N 

89068, "METHOD AND SYSTEM FOR FLDOBLE 

HAVIOR OF SOFTWARE APPLICATION," bearing at- 
torney docket number N.89070, and "METHOD Awn 
SYSTEM FORGENERATINGFLEXIBLeSe^D 
SeSh-: APPEAR/SicETS 
BEHAVIOR, beanngattomeydocketnumberN. 89072 
all of which are incorporated by reference 
[0002] This invention relates generally to computer 
syst^ and software applications therefor and more 
specrfically. to timing, control, and execution of sofhvare 
applications. 

fS. ^! ™P™ved price-perfomiance of digital 
t^hnologyhasmadeposslble the advent Of dlgltalme- 
dia broadcasting. The reduced cost of microprocessors 
^ rtaLmemofy,^ndjelatedjlevjces,h a^adP it pnco,,' 

^l?^Hi*xomputers.in.set,top5oxes^ 
lated-defl&fi.tn r<«v»*,i u..- .. . ' ^ 



a ed devices-to receive digital cable or digital satellite 
^^^'^al^BeWeeode^hose-eignaL.o^ 

Visual programs. 

K '^^r. °' broadcasting 
and the proliferation of STBs also makes it possible to 
enhance viewere' television experiences. In addition to 
transmitting video and audio data, digital media broad- 

STBs with Which users can interact By analogy, users 

it j f r '^'^ transmtted via 

the digitalmediabroadcastthewayacomputer user can 
interact with data available over the Intemet. For exam- 
ple digital television subscribers are already acquainted 
wrth an electronic program guide (EPG) function which 
enables users to select and tune to particular programs 

and^orreadabouttheprogramsbelngshowna'tpSen 
or at later points in time. 

[0005] The EPG is a rudimentary example of an inter- 
active television application exploiting the capability of 
the digital broadcast medium to transmit additional con- 
tent and for the STB to execute this additional content 
TTie computer functionality of STBs also makes other 
interactnre television applications possible. With appro- 
pnately configured STBs, users potentially can play 
along with game shows, take tests in on-line distance 
learning courses, bid in on-line auctions, and otherwise 

actively engage theoontentbeing broadcast. STBs thus 
allow users to have highly interactive television experi- 

Sc?"' T. °' '=°"'^"t application 

whch IS desirable in such a broadcast environment is 
one Which provides functionality synchronized with vid- 
eo or audio content on a broadcast channel. In the ex- 
amples of interactive game shows, on-line distance 



learning testing, and on-line auctions, it would be highV 

sJI^Jk .f''^°*^'°^'^'«'^«^^'''«°andaudio 
5 b!^^r^''"T°''''''''^°^'*''""9«PP«*'«tion-spe<^fe 

sntTl'". '^•'^^'^""S the appl4S, 

specrfic behaviors with video and audio streTs 
presents a number of concerns. These concemsTd 
conventional responses t6 these concerirTre" de 

« .^^r^'riedt" ' f "'"^"'""^ ^^'^^ 

B^corporated by reference. Nonetheless, at least two of 
these concerns merit repeating ■^sitwoof 

to sm, "^""'"^ ""P""^""" *P^'«° behaviors 
to STBs conventionally involves transmitting applica- 
bon-specific program code to the STBs. The tranS 

d.um. just as video and audio data are trensrnitted 
Downtoading such applications invoh.es SmS 
poten ally many data packets, comprising v^tTum 
bereofdatablocks,foreachapplicatlon.AconSmaris- 
f"f '"^ '^^ta for the numerous 
Channels being transmitted leaves lime banZth fi 

^dte-necessan^4o-trensmir-i,r^^^K^^^ 
specilic^}rogrammino-to-«nn„^ r>,^„-,-— ^r" 



:. .„ ..J "•^^"'""""""-' arge Dodies-of -apDlinfltinn 

[0008] Second, synchronizing behaviors of the aDoli- 
cation specific programming with a series o evenTs n 
ttisrr'"''"';°'^"""''^<^'^'-«Applicat2; 
il^^hT!""""'"^ asynchronous trigger sig- 
nals embedded in the broadcast signal These a^ 
chronous trigger signals can be difficult to delter aS-" 
rately When data blocks carrying such triggere musTJi 
« multiplexed with so much other video a^ auJo date 
transmitted in the medium. Further, these Bme-d^J 
encies present particular concerns when a user enqaa- 
es a program already in progress and may have mS 
a timing synchronization event at the outeet of tJ^ pro 
« gram. Correlating the application-specific prooramminn 

rrosir^-'-^-vbetro^s 

[0009] Developers of interactive programming or oth- 

content to go along with video and audio data are sut 
ably concerned With how toprovideandcorrelate'mr^q 
ofsudicontent. It ishighly desirable todevelop 
n STBs o reduce data transmission needs and to a«ow 

50 L iH Ttr/ corresponding 
ly, It IS highly desirable for developere to be able to Z 
and modify applications to ensure that they will ope S 
appropnateiy in the STB environment, and that the S 
ng of events in such applications is appropriate s l 
lariy. * is highly desirable to be able to emuiatL ^og^am 

of the applications to make sure that the applfeations wi« 
n.n properly on STBs . It is important that SeveCere be 
able to review importantportions Of applications, and be 
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able to make desired modifications. 
[0010] Thus, there are unmet needs in the art for fa- 
cilitating efficient transfer of data in a broadcast environ- 
ment that allows for accurately synchronized behavior 
by an application, as well as for testing, monitoring, and 
modifying such applications prior to transmission to en- 
sure desired execution by users. 
[001 1] Embodiments of the present invention enable 
provide support for controlling program appearance and 
behavior during execution of a computer system. Be- 
havioral attribute changes are correlated with state 
changes which are associated with a list of triggering 
events which initiate the behavioral changes. As a re- 
sult, applications can be generated with provision of da- 
ta In the form of collections of state changes and event 
lists without having to generate or provide custom ap- 
plication code. 

[001 2] More specifically, embodiments of the present 
invention provide a method, a computer readable medi- 
urn, and a data system for controlling computer system 
operation during execution of the computer system . Us- 
ing embodiments of the present invention, at least one 

^^attribute.change.of-axoniputer-system-to-occur-during— 

— execution-of-thexomputer-sys^ 

" trib'ute charige is^sso'ciated with" a state change occijr~ " 

rin g i n th e c emputef^ystem-suc h - th at the-s tat e chan g e 

initiates the attribute change. Index points are identified 
in the execution of the computer system indicative of 
when the state change Is to occur. These index points 
are associated with appropriate state changes. Upon 
execution of the computer system, reaching events as- 
sociated with the index points triggers the associated 
state changes and, thus, the associated attributes are 
changed automatically upon reaching the appropriate 
index .points. 

[001 3] Embodiments of the present invention suitably 
operate in a broadcast environment where execution of 
the computer system is correlated with a media pro- 
gram, and the media program and data concerning at- 
^ tribute chaniges and index points are broadcast via dig- 

Ital transmission. Embodiments of the present invention 
also suitably operate in an emulation environment 
where the media program and data concerning attribute 
changes and index points are loaded into an emulator 
where their presentation can be tested, monitored, and 
revised If necessary. 

[0014] In accordance with further aspects of the In- 
vention, the index points are collected into index tables, 
and the state changes are collected in state tables. The 
index table and the associated state table can be mod- 
ified, updated, and/or transmitted independently of one 
another, and can be separately or sequentially loaded 
from a data stream containing the Index table and the 
state table. The index table and state, table can be re- 
ceived from a broadcast network and viewed at that time 
or stored for later execution. The Index table and state 
table also can be stored directly on a system and exe- 
cuted in an emulation environment to monitor applica- 



tion behaviors according to an emulated passage of 
time. The emulated passage of time can be modified to 
scrutinize resulting behaviors to allow for monitoring of 
application parameters, and to allow for modification of 
s the Index table and state table to adjust application be- 
haviors. 

[001 5] The preferred and alternative embodiments of 
the present Invention are described in detail below, by 
way of example, with reference to the following draw- 
10 ings, in which:- 



FIGURE 1 shows a timeline of event triggers driving 
an index table and a state table for controlling an 
interactive software application according to an em- 
bodiment of the present invention; 
FIGURE 2 is a flowchart of a routine for executing 
state changes and index points in an index table ac- 
cording to an embodiment of the present invention; 
FIGURE 3 is a flowchart of a routine for emulating 
state changes and Index points In an Index table ac- 
cording to an embodiment of the present invention; 
FIGURE 4 is a Is a set of windows for reviewing and 
-revisingzattfibute.changes-and-index^points ; 
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-lilGURE^5-is-a-biock-diagram:of-an:environment for.-- 
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creating, disseminating and using applications'us- 

- ing an en tbudiinenl of the present Invention; 

FIGURE 6 is a block diagram of an Interpreter exe- 
cuting an application using state changes and Index 
points in accordance with an embodiment of the 
present Invention; and 

FIGURE 7 is a block diagram of a data processing/ 
media control system for executing or emulating ap- 
plications according to an embodiment of the 
present Invention. 



[0016] By way of overview, embodiments of the 
present invention provide a method, a computer reada- 
ble medium, and a data system for controlling computer 
system operation during execution of the computer sys- 
^0 tern. Using embodiments of the present Invention, at 
least one attribute change of a computer system to oc- 
cur during execution of the computer system is Identi- 
fied. The attribute change Is associated with a state 
change occurring in the computer system such that the 
45 state change initiates the attribute change. Index points 
are identified in the execution of the computer system 
indicative of when the state change is to occur. These 
index points are associated with appropriate state 
changes. Upon execution of the computer system, 
50 reaching events associated with the index points trig- 
gers the associated state changes and, thus, the asso- 
ciated attributes are changed automatically upon reach- 
ing the appropriate index points. Embodiments of the 
present invention suitably operate in a broadcast envi- 
55 ronment where execution of the computer system is cor- 
related with a media program, and the media program 
and data concerning attribute changes and index points 
are broadcast via digital transmission. Embodiments of 
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he present invention also suitably operate in an emula- 
tion environment where the media program and data 
concerning attribute changes and index points are load- 
ed into an emulatorwhere theirpresentation can be test- 
ed, monitored, and revised if necessary. 
mm FIGURE 1 shows exemplary Interaction be- 
tween a video timeline 102. foraprogram with identified 
events 104. 106, 108, and 110, an index table 112 a 
plurality of state changes collected In a state table 1 14 
and a program display 116 in one embodiment of the 
invention that is given by way of a non-limiting example 
The operation of the state changes stored in the state 
table 114, the operation of the index points contained in 
the index table 112, and a facility for generating such 
control information as manifested in the tables 112 and 
114 are further described In the concurrently-filed co- 
pending patent applications entitled "METHOD AI^D 
SYSTEiM FOR AUTOMATIC CONTROL OF GRAPHI 

EXECUTION, bearing attorney docket number N 
89068, "METHOD AND SYSTEM FOR FLEXIBLE 
TIME-BASED CONTROL OF APPEARANCE AND BE- 
ijAVIQB-0.E-S.QElWA BEJLeeHCATIOM • h»«rinn 

tomeV dnnieot hnmkar M snTvn-c — — -1-^ — 
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20 



c 



==lo.jiejr_aocket nurnber N.S9ri7Drari5a:"METHOthO\rtT 
SYSTEMFORGENERATINGFLEXIBLETIME-BASED 

BEHAVIOR," bearing attorney doclcet number N.89072 
respectively, the contents of which are hereby incorpo- 
rated by reference. 

[0018J The video tmellne 102 depicts foursequences 
in the program, including an introduction 124 Part 1 
126, Part2 128, and Parts 130. The beginnings of these 
sequences 124. 126, 128, and 130 are identified as 
events 104, 106, 108, and 110, respectively, to be used 
in correlating presentation of additional content The 
events 104, 106. 108. and 11 Oara thus associated with 
index points 174. 176. 178. and 181 and collected in the 
index table 112 in sequential order. In turn, each of the 
index points 174, 176, 178, and 181 are associated with 

tTVT^:r!.'^- ^"^ '^P^^^^fy. In the state 
table 114. The state table 114 also lists three program 
attnbutes 164, 166, and 168 which control appearance 
and behavior of the program as shown on the program 
display 116. As a result, as the video timeline 102 
progresses with presentation of the program, each of 
he events 104, 106, 108, and 110 are reached. Reach- 
ing each of these events 1 04, 1 06, 1 08, and 1 1 0 triggers 
index points 174, 176, 178, and 181, respectively, in the 
index table 112. Reaching index points 174, 176 178 
and 181 in the index table 112 triggers state changes to 
states 144, 146, 148, and 150, respectively, thereby po- 
tentially changing each of the program attributes ap- 
peanng on the program display 116. 
tool 9] More specifically, as represented on the video 
timeline 102, the program commences at a beginning 
associated with event 104. The event 104, which occurs 
at frame 0" at the beginning of the program, is associ- 
ated with index point 1 74 and state 144, which is a first 



stetein the state table1l4. At the state 144,aprogram 
attnbu^e-btn.visible-180issettoT-182fortme,the?e 
by making a button 1 84 appear on the program display 
16. A program attribute "image.fiie" 186 is set to "a" 
188, thereby making an associated image 190 appear 
on the program display A program attribute 

ground on the program display lie 

to he video timeline 102 and reaches the event 106 
Which occurs at "frame i- at the beginning of a next seg- 
ment of the program. The event 106 occurs at "frame i" 
and IS associated with index point 176 and state 146 
^^t:JT- '^^""'"^ ^^"^ '""^^ P°'"t 176 auto. 
2e S« .h''"^ ""^"9" 146. At the 

state 146, the program attribute "btn.visibie" 180 is set 
to "F" l96forfalse. thereby making the button 184 dfe 
appear from the program display 116. The program at- 
mbute "image.flle" 1 86 is set to "b" 1 98, making an as- 
sociated image (not shown) appear on the progL dis- 
play in the place of image 190. The progL attribute 
Wn.color- 192 is set to "8" 199, aga' chan^g ^ 
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35 



reache d, index pom ts 178 and 18T:^spiSvely, trigger 

T ^^°:/^^P^'^«'y. thereby changing the appear- 
ance and behavior of the program as evidenced by the 
appearance of the display screen 116. In sum, Len 
events logged as index points are reached during exe- 
cution of the program, states changes associated with 

Sbrzr^^''^=^^^-'"^^«"^-°'^^'««^- 

[0021] It will be appreciated that execution of an ap- 
plication as described by the non-limiting example ad- 
vantageousfy can be perfomted upon actual exLtion 
of the application at a viewer/User location or in emulat- 

Zl!T^T" "'"^^ P""^'" ^■^s^'nina- 

don of tt,eapplK:ation. In fhefomiercase, the application 
might be executed by a STB which first loads the state 
flange and Index point infomiation, then executes the 
state changes according to the index points. In the latter 
case, the application might be executed on a computer 

cTl ' l''.^™ ^^'"''g the 'tate 

change and index point infomiation and allowing pro- 
gram developers or editors to monitor how the applica- 
tion will execute in a view/user environment. FIGURES 
2 and 3 show routines for how such execution and em- 
ulation, respectively, can be perfomied according to em- 
bodiments of the present invention 
[0022] FIGURE 2 shows a flowchart of a routine 200 
for executing an application implemented using a series 
of state changes and index points collected in a state 

a STB suitably coupled with a televteion or other moni- 
tonng equipment. TTie routine commences at a block 
210. At a block 220, a state table is loaded, the ml 
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table listing the state changes which define the behav- 
iors of the application. At a block 230, an index table is 
loaded. The Index table lists the Index points which trig- 
ger state changes listed in the state table. It will be ap- 
preciated that either the loading of the state table at the 
block 220 or the loading of the index table at the block 
230 may be perfonned first. In addition, if eitheracun^ent 
state table or a cun^ent index table is already resident in 
the STB, only that table which was not previously loaded 
is loaded in the routine. Thus, if current versions of both 
tables already are loaded in the STB, neither need be 
loaded again. Itwili be appreciated, therefore, that if one 
of the tables is updated or revised, only the revised table 
would have to be received. Accordingly, only the revised 
table would have to be transmitted, thereby saving on 
bandwidth. 

[0023] it will also be appreciated that, while the at- 
tribute changes are correlated into state changes and 
collected in a state table, infomnation determining the 
attribute changes also could be collected In object-spe- 
cific lists and other fomis. Similarly, while index points 
have been collected into index tables for purposes of 

sjionzJimlting^xaiiipIe,4ndeX:Pojptsalso^uld-be^ol-— 
jected into.ind^^ 



ed by reference. If index points are not being reached 
at the decision block 250, the routine checks to see if 
the last Index point has been reached at a decision block 
270. If not. the routine loops to the decision block 250 
5 until an Index point Is reached at the decision block 250 
or a last index point is reached at the decision block 270. 
If a last index point is reached at the decision block 270, 
the application ends at a block 280. 
[0026] FIGURE 3 is a flowchart of a routine 300 for 
10 emulating execution of the application. The routine 300 
commences at a block 310 and, like the routine 200 
(FIGURE 2) starts by loading a state table at a block 320 
and loading an index table at a block 330. It will be ap- 
preciated that loading of either table could proceed first. 

IS It will also be appreciated that, in an emulator environ- 
ment, the tables likely will be loaded from a local storage 
device rather than received froni a broadcast medium. 
For a non-limiting example, the routine 300 suitably is 
executed on a computer on which the state table and 

20 Index table have been stored on a hard disk or other 
recording medium. The computer-suitably may be the- ■ 
same device on whrch the application and tables previ- 

Qusly-have-beenrC^eatedT ^ 



clarity ahd'bTevity-of-explanationrthe^ 



used to r e fer to a colle c t i on of s t ate cliange/att i ibule 
change data, and the term index table is used to refer 
to a collection of index point data. 
[0024] Once the state table and index table have been 
loaded, program sequence execution is initiated at a 
block 240. If the viewer or user has engaged the pro- 
gram at the time of program broadcast, Initiation of the 
program sequence at the block 240 could be triggered 
at a broadcast site. Alternatively, if the viewer or user 
later engages the program time from a recording device, 
that same program initiation signal embedded, in the 
broadcast suitably initiates program execution upon re- 
playing of the broadcast. In any case, initiation of pro- 
gram execution then begins a passage of time or a se- 
quence of events which con-espond with the Index 
points received in the index table, thereby facilitating 
program execution. 

[0025] As the program executes, video and/or audio 
infomiation is decoded and presented to the user, along 
with any continuous content stored in the data included 
with the program. When an index point is reached at a 
decision block 250, associated state or aUribute chang- 
es are made at a block 260. Again, the operations of 
state tables and index points are further described In the 
concurrently-filed, co-pending patent applications enti- 
tled "METHOD AND SYSTEM FOR AUTOMATIC CON- 
TROL OF GRAPHICAL COMPUTER APPLICATION 
APPEARANCE AND EXECUTION," bearing attorney 
docket number N.89068, and "METHOD AND SYSTEM 
FOR FLEXIBLE TIME-BASED CONTROL OF AP- 
PEARANCE AND BEHAVIOR OF SOFTWARE APPLI- 
CATION," bearing attorney docket number N.89070, re- 
spectively, the contents of which are hereby Incorporat- 



,-[00271_Oncethe.tables:are.loaded,.at.ablodk.340^ex- 

25 ecution parameters-are monitored. These" parameters" 

suitably include utilization of memory or other resourc- 

es, execution time and other-time related infomnation, 
or other parameters. Allowing an operator to monitor 
these parameters allows the operator to further ensure 
30 proper run-time behavior once the application is broad- 
cast. 

[0028] At a block 350, the operator controls an emu- 
lated passage of time . At the block 350 , the operator can 
let the application execute according to an ordinary tim- 

35 ing sequence that might be expected in a broadcast en- 
vironment. Alternatively, the operator can execute the 
application in slow motion at a slower execution rate 
than would nonnally be expected. Alternatively, the op- 
erator can execute the application onetime unit or frame 

40 at a time. Alternatively, the operator can execute the ap- 
plication more rapidly than would be expected in a 
broadcast situation. Also, the operator can move back- 
ward as well as forward through application execution 
to review to his or her satisfaction operation of the ap- 

45 plication. 

[0029] As the passage of time Is emulated, at a deci- 
sion block 360 it is detennined if an index point has been 
reached. Once an index point has been reached, at a 
block 370 the associated behavioral changes, including 

50 state changes, are executed. At a decision block 380, 
the operator detemnines whether the behavioral chang- 
es resulting from the interaction of the state changes in 
the state table, the index points in the index table, and 
the emulated passage of time yield an acceptable result. 

55 If the result is acceptable, the operator continues with 
the control of the emulated passage of time at the block 
350 to further emulate the application. If the.result is not 
acceptable, at a decision block 380, the operator can 
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revise the state changes in the state table or the index 
points In the Index table to modify the execution of the 
application.Aftermaldng such modificatons, the routine 
resumes with the operator'scontroloftheemulated pas- 
sage of time at the block 350 to further emulate appli- 
cation execution, in this way, the operator can execute 
an aspect of the application that was just revised to de- 

temnineifthersvisionsareaoceptable.Theoperatorcan 
continue with the emulated control of time at the block 
350 indefinitely, or the operator can choose to terminate 
emulation of the application at any time (not shown) 
[0030] FIGURE 4 shows an exemplary embodiment 
of a monitoring environment 470 in which the routine of 
FIGURE 3 suitably is implemented for emulating execu- 
tion of the application. The emulating environment in- 
cludes a viewing window 472 for showing the program 
including attribute changes 474 generated. An applica- 
tion icon 476 is situated on the screen to signal to a view- 
er that the object on the screen mari<ed by the applica- 
tion icon 476 is of interest to him or her. An application 
icon 476 can be used, for example, to signal a viewer 
interaction opportunity, such as the user having a 

f ^'?''°'P^"fPfa°8e.theJtMTim aiJ(ed.withiheaDDiieatinn 
■con 47<i U"uug!iljiejggrojjMsj>^^ 

ifitntei ^rtntrSI A;c~;Ci.i»i:jr:_*rL- — - . — .- 
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mg environmentSIO. the application is disseminated in 
a broadcast environment 520. As previously described 
digital applications can be transmitted via digital cable 
fransmission. digital satellite transmission, or also may 
be transmitted via the Internet or other means. The ap- 
plication, including the state changes and the index 
points (not shown) are received at an STB 530 where 

the STB 530. The interpreter 540 receives and process- 
es the state changes and Index points, such as in the 
manner described in the routine 200 (FIGURE 2) The 
output of the interpreter 540 Is presented to a user or 
viewer via a monitor 550, which suitably is a television 
an audiovisual monitor, a computer, or another devfc^ 
equipped to process the application. 
[0033] As previously described, Interaction with an 
apptemion is possible through the inteipreter 540 and 
me STB 530. Applications suitably allow for interaction 
not only at the STB 530 level, but also at a broadcast 
site level, as in the examples of on-line testing, on-line 
auctions,.and similar.applications. in such cases 



mote control. Ah-appI(catroh~wrmj6w-allowi:478~aH^^^ 
for toxt and e o mmond d ii vu in ui i ipulation of c o d e 



turn path 560 allows for user/Viewer input back to the 

-broadcast.^ite^nict Man.be.tecon>orated^v^,d »n.,.i. 

_transni!SSLonrDrocessor- Rvn^nx^^^^^w.^ * , . ~ 



— T- : — , '''^^-mwtiJUiama -ov-SLTaynfimin — 

- «~-?^^^^*^-°^^°^'®^°"slv.e to.user/viewec - " 
input. For-e xample.tfthe-apptoTiOTTnviteTv1^e^^^ 

P'^" ill an uii li i i u pu ll , yai li eied by ilie In- 
terpreter 540 can be transmitted via the STB 530 over 
the return path 560 to the dynamic transmission proc- 
essor 570 where the poll can be tallied. Results can be 
relayed viaallve host or, if the audiovisual portion of the 
program Is prerecorded, the results could be encoded 
as data and processed by the interpreter540 in the STB 
530 0 be presented to the user It will be appreciated 
that the return path 560 for digital STBs suitably is a 
phone line through which users order pay-per-view 
movies and through which other transactions are proc- 
essed. The return path 560 could be a phone line, a re- 
turn band through the digital broadcast medium, or an- 
other implementation. 

[0034] it will be appreciated that the application could 
be emulated by a system capable to mimic the functions 
of the STB 530 and the Interpreter 540. In an emulation 
environment, the application would not be broadcast 
but instead It would be loaded from a local medium and 
executed. Functions of the dynamfc processor 570 in 
response to user actions could be simulated locally as 

W6i|, 

[0035] FIGURE 6 shows a block diagram of one pres- 
ently preferred embodiment of the inteipreter 540 The 
interpreter 540 processes index points 802 and state 
changes collected In a state table 606 via an execution 
model 604. The executfon model 604 contains standard 
function calls to intenjret functions invoked in the state 
table. The execution model 604 also is responsive to the 
index points to sequence through the state table as dic- 
tated by the application behaviors encoded in the state 
table 606. In one presently preferred embodiment, as 
the execution model 604 steps through each state 



— -..WW iiiaiu ^ju i uuu ii u i coqe as* 

sociated with the attribute changes, if the developer de- 
sires to make changes beyond or in addition to those 
that can be made through manipulation of the tables A 
status window 480 textually presents a list of functions 
being executed to present the attribute changes shown 
so that the developer can monitor what is causing the 
attributes shown on the screen. 
[0031] For testing interactive programs, a recording 
window 482 anda remotewindow 484 are provided The 
developer can simulate how a viewer or user of an in- 
teractive program might interact with a STB or other de- 
vice to engage the program. For example, if a program 
such as a coolcing show offers opportunities to purchase 
merchandise, the developer can click on a screen button 
485 to simulate a user making a choice on a keypad As 
a result, this allows the developer to determine how well 
interactive attribute changes he or she created will func- 
tion. Similarly, the recording window 482 records actions 
of the simulated user to allow the interactive program to 
be played back to review how well the interactive func- 
tionality operated. A resource monitoring window 486 
also allows the developer to monitor system resource 
usage, such as memory usage, to detemilne whether 
the program meets practical constraints and objectives 
[0032J FIGURES shows a block diagram of an exem^ 
plary system embodiment of the present invention in a 
production and broadcast environment. Applications 
are created in an authoring environment 510. Suitably 
the applications are emulated there according to the 
routine 300 (FIGURE 3) to detemiine If they will execute 
satisfactorily in a broadcast environment. Once the ap- 
plication has been satisfactorily authored In the author- 
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change In the state table, specified operands in the state 
changes are pushed onto an execution stack 608. A 
byte code interpreter 61 0 then retrieves these operands 
which, for non-limiting examples, can be values or ad- 
ditional function code. The byte-code Interpreter 610 
then implements the behaviors that were encoded in the 
state table 606. triggered by the Index points 602, and 
directed by the execution model 604 to the execution 
stacl< 608. 

[0036] FIGURE 6 does not show a clock or other syn- 
chronizing device, although it will be appreciated that 
some Indication of passage of time, whether a frame 
number count, elapsed time, or other such indicator is 
desired to drive execution of the computer system. For 
stored programs and data, a system clock, reading of 
frame or other sequence numbers, or similar means can 
be used to drive execution of the system in a standalone 
or emulation environment. In a broadcast environment, 
time or f rarine indicators would be broadcast or a system 
clock suitably synchronized to a broadcast time or coun- 
ter could be used to drive execution of the computer sys- 
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scribed in connection with the foregoing example, meth- 
od flowcharts, block diagrams, and co-pehding patent 
applications incorporated by reference. User input or 
emulated user input can be retumed by the Input device 
controller 726 through the system controller 71 8 to the 
CPU 71 6 for processing. In turn, the CPU 71 6 transmits 
the information through the input/output controller 710 
to the network or storage 706 as appropriate. 
[0040] While the prefen'ed embodiment of the Inven- 
tion has been illustrated and described, as noted above, 
many changes can be made without departing from the 
scope of the invention. Accordingly, the scope of the in- 
vention is not limited by the disclosure of the preferred 
embodiment. Instead, the invention should be deter- 
mined entirely by reference to the claims that follow. 



Claims 

1 . A method for controlling computer system operation 
during execution of t he computer system, the mgth- 



FIGURE 7 sho ws . a conriputer system 7QQ, 



od comprising: 



— --— which^OQtd^b^lfTtheiformrofraimedia controller, STB, 



--"identlfy.ing_aneastj)neH^ 



:~or:emulatorJstation:operable:for-usirig-embodim 
the present i nvent i on. The computor syotom 700 is op - 



-eomputer-system-to-occurduS^ 

t he c o m p u t er system; 

associating the attribute change with a state 
change in the computer system such that indi- 
cation of the state change triggers the attribute 
change; 

identifying an Index point In the execution of the 
computer system indicative of when the state 
change is to occur; 

associating the Index point in the computer sys- 

35 tem with the state change; and 

triggering the state change In the computer sys- 
tem upon reaching the index point associated 
with the state change, such that the attribute is 
changed automatically upon the index point be- 

^0 ing reached. 

2. The method of claim 1 , wherein each of the index 
points associated with the state changes Is collect- 
ed in an index table. 

The method of claim 2, wherein the index table in- 
cludes a first column listing the index points and a 
second column listing the associated state chang- 



4. The method of claim 2 or 3, wherein the index table 
is ordered by sequential occurrence of the index 
points. 

55 5. The method of any one of the preceding claims, 
wherein state changes for attribute changes are col- 
lected in a state table, the state table listing the at- 
tribute changes along a first dimension of the state 



erable for controlling a display 702, such as a television, 
and an audio subsystem 704, such as a stereo or a loud- 
speaker system. The computer system 700 receives In- 
put from a network or storage 706 in the case of the STB 
or the emulator station, respectively. The computer sys- 
tem 700 also receives user input from a wired or wireless 
user keypad 708, which may be In the nature of a STB 
remote, a computer keyboard, or another input device. 
[0038] The computer system 700 receives input via 
an input/output controller 71 0, which directs signals to 
and from a video controller 712, an audio controller 714, 
and a central processing unit (CPU) 71 6. In the case of 
a STB, the input/output controller 71 0 suitably is a mul- 
tiplexer for routing video data blocks received to a video 
r ; controller 712 in the nature of a video decoder, audio 

data blocks to an audio controller 714 in the nature of 
an audio decoder, and for routing other data blocks to a 
CPU 71 6 for processing. In turn, the CPU 716 commu- 
nicates through a system controller 71 8 with input and 
storage devrces such as read only memory (ROM) 720, 
system memory 722, system storage 724, and input de- 
vice controller 726. 

[0039] The computer system 700 shown in FIGURE 
7 thus can receive state changes collected In state ta- 
bles and index points collected in index tables and pass 
them through the Input/output controller 71 0 to the CPU 
716 where they will be processed through the system 
controller 718, suitably in response to user input gath- 
ered through the user keypad 708 and the input device 
controller 726. The state changes collected in the table 
can then be executed and/or modified as triggered by 
the Index points in the Index table as previously de- 
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6. 



table and the state changes along a second dimen- 
sion of the state table such that as each cun-ent 
state change is reached, each attribute change as- 
sociated with the cun^ent state change is triggered. 

The method of claim 5. wherein the index table and 
the state table are revisabie Independently of each 
other such that changing one of the Index table or 
the state table does not necessitate changing both 
the index table and the state table. 



10 



7. The method of claim 5 or 6, further comprising one 
of sequentially loading or separately loading the in- 
dex table and Its associated state table from a data 
stream received by the computer system. is 

The method of any one of the preceding claims, 
wherein the execution of the computer system is 
based on a sequence of state changes and Index 
points received from a broadcast network. 20 



8. 



9. The method of claim 8, wherein the sequence of 
state.xhanges-and,theJndex-points-are-received- 



Ifrom tne-Droadcast netwo rk and fitnroH for «?iih«?r ~ - ^ . — = ^ 

- - - - -quent executiori^z:^- 



17. A system for controlling computer system operation 
during execution of the computer system, the sys- 
tem comprising: 

a user Interface including: 

a first component configured to identify at 
least one attribute change of a computer 
system to occur during execution of the 
computer system; and 
a second component configured to associ- 
ate the attribute change with a state 
change in the computer system such that 
Indication of the state change triggers the 
attribute change; and 

a processor including: 



a third component configured to identify an 
index point in the execution of the compu- 
— tecsystemJndicatLve-DL wh en tha 
change is to occur; 

ra-fourth-component-configured^to-as soci- 



change;-and 



35 



40 



10. The method of any one of the preceding claims, 
wherein the execution of the computer system Is 
based on a sequence of state changes and index 
points stored on the computer system. 30 

11. The method of claim 10. wherein the sequence of 
state changes and Index points Is revisabie for con- 
trolling the execution of the computer system during 
the actual passage of time. 

12. The method of any one of the preceding claims, 
wherein passage of time for controlling the execu^ 
tion of the computer system is emulated and is ma- 

nipulable for emulating actual execution of the com- 
puter system during actual passage of time. 

13. The method of claim 12, wherein parameters for 
emulating actual execution of the computer system 
are monitorable. 

14. The method of claim 13. wherein the parameters 
include execution running time. 21 

15. The method of claim 13 or 14. wherein the param- so 
eters include memory usage. 

16. A computer readable medium for controlling com- 
puter system operation during execution of the 
computer system, the computer readable medium ss 
comprising computer program code means for car- 
rying out the method defined In any one of claims 1 

to 15. 



4S 



a nnii uoiiipoiieiii c onfigured to trigger the 
state change In the computer system upon 
reaching the index point associated with 
the state change, such that the attribute is 
changed automatically upon the index 
point being reached. 

18. The system of claim 17. wherein the processor fur- 
ther includes a sixth component configured to col- 
lect in an index table each of the index points asso- 
ciated with the state changes. 

19. The system of claim 18, wherein the sixth compo- 
nent is further configured to generate the index ta- 
ble with a first column listing the index points and a 
second column listing the associated state chang- 
es. 

20. The system of claim 18 or 19, wherein the sixth 
component is further configured to order the index 
table by sequential occurrence of the index points. 



The system of any one of claims 1 8 to 20. wherein 
the processor further includes a seventh compo- 
nent configured to collect in a state table state 
changes associated with the attribute changes, the 
state table listing the attribute changes along a'first 
dimension of the state table and the state changes 
along a second dimension of the state table such 
that as each current state change Is reached, each 
attribute change associated with the current state 
change is triggered. 
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22. The system of claim 21 , wherein the processor fur- 
ther Includes an eighth component configured to in- 
dependently revise the index table and the state ta- 
ble such that changing one of the Index table or the 
state table does not necessitate changing both the s 
Index table and the state table. 

23. The system of claim 21 or 22, wherein the processor 
further Includes a ninth component configured to 
perfonn one of sequentially loading or separately 
loading the index table and its. associated state ta- 
ble from a data stream received by the computer 
system. 

24. The system of any one of claims 1 7 to 23, wherein ^5 
the processor further includes a tenth component 
configured to receive the sequence of state chang- 
es and Index points from a broadcast network. 



25. The system of claim 24, wherein the tenth compo- 
nent is further configured to store the sequence of 
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receiving a media program including at least 
one of video content and audio content; 
receiving related data comelated with the media 
program for controlling computer system exe- 
cution during presentation of the media pro- 
gram; 

identifying in the related data an attribute 
change to occur during presentation of the me- 
dia program; 

Identifying in the related data an index point as- 
sociated with the attribute change for triggering 
the attribute change upon detecting occun-ence 
of the initiating event during presentation of the 
media program; 

presenting the media program; and 
detecting the initiating event associated with 
the Index point and triggering the attribute 
change associated with the Index point. 

33. A computer readable medium for controlling execu- 
tlon of a computer .system during presentatjofl-o^a— 



state changes and the Index points for subsequent 

execution. 



media program, the computer readable medium 
=^0Bapflsmg4= 
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-267^he'system~of^ni^e"of elai 

tho proc e ssor furth e r i nclud e s a n e le ve n t h c o mp o - 

nent configured to control execution of the compu- 
ter system based on a sequence of state changes 
and Index points stored on the computer system. 

27. The system of claim 26. wherein the processor fur- 
ther includes a fourteenth component configured to 
revise the sequence of state changes and index 
points tQ modify execution of the computer system 
during actual passage of time.' 

28. The system of any one of claims 1 7 to 27, wherein 
the processor further includes a twelfth component 
configured to emulate passage of time and to ma- 
nipulate the passage of time thereby allowing for 
emulating actual execution of the computer system 
during actual passage of time. 

29. The system of claim 28, wherein the processor fur- 
ther Includes a thirteenth component configured to 
monitor parameters for emulating of the actual ex- 
ecution computer system. 

30. The system of claim 29, wherein the parameters in- 
clude execution running time. 

31. The system of claim 29 or 30, wherein the param- 
eters include memory usage. 



32. A method for controlling execution of a computer ss 
system during presentation of a media program, the 
method comprising: 



-first-computer program-codemeanff^forre(?elv=" 



Ing a media program including ai least one or 
video content and audio content; 
second computer program code means for re- 
ceiving related data correlated with the media 
program for controlling computer system exe- 
cution during presentation of the media pro- 
gram; 

third computer program code means for identi- 
fying in the related data an attribute change to 
occur during presentation of the media pro- 
gram; 

fourth computer program code means for iden- 
tifying in the related data an index point asso- 
ciated with the attribute change for triggering 
the attribute change upon detecting occurrence 
of the Initiating event during presentation of the 
media program; 

fifth presenting the media program; 
detecting the initiating event associated with 
the index point and triggering the attribute 
change associated with the index point. 



34. A system for controlling execution of a computer 
system during presentation of a media program, the 
50 system comprising: 



a receiver Including: 

a first component configured to receive a 
media program including at least one of 
video content and audio content; and 
a second component configured to receive 
related data correlated with the media pro- 
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gram for controlling computer system exe- 
cution during presentation of the media 
program; and 

a processor including: 

a third component configured to identify In 
the related data an attribute change to oc- 
cur during presentation of the media pro- 
gram; 

a fourth component configured to identify 
in the related data an index point associat- 
ed with the attribute change for triggering 
the attribute change upon detecting occur- 
rence of the initiating event during presen- 
tation of the media program; 
afitth component configured to present the 
media program; and 

a sixth component configured to detect the 
initiating event associated with the Index 
point and trigger the attribute rhap j^^^ 
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gram; 

third computer program code means for identi- 
fying In the related data an attribute change to 
occur during presentation of the media pro- 
gram; 

fourth computer program code means for iden- 
tifying in the related data an index point asso- 
ciated with the attribute change for triggering 
the attribute change upon detecting occurrence 
of the initiating event during presentation of the 
media program; 

fifth emulating presenting the media program; 
and 

detecting the initiating event associated with 
the index point and triggering the attribute 
change associated with the Index point. 

A system for emulating execution of a computer 
system during presentation of a media program, the 
system comprising: 



sociated with the Index point. 




-systemduHng presentation-ef^T^i^liip^^gi^ 
m otli o d c um p i isii i g. 
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loading a media program including at least one 
of video content and audio content; 
loading related data correlated with the media 30 
program for controliing computer system exe- 
cution during presentation of the media pro- 
gram; 

identifying in the related data an attribute 
change to occur during presentation of the me- 35 
dia program; 

identifying in the related data an index point as- 
sociated with the attribute change for triggering 
the attribute change upon detecting occurrence 
of the initiating event during presentation of the 40 
media program; 

emulating presenting the media program; and 
detecting the initiating event associated with 
the index point and triggering the attribute 
change associated with the index point. 4s 

36. A computer readable medium for emulating execu- 
tion of a computer system during presentation of a 
media program, the computer readable medium 
comprising: 



first computer program code means for loading 
a media program including at least one of video 
content and audio content; 
second computer program code means for ss 
loading related data correlated with the media 
program for controlling computer system exe- 
cution during presentation of the media pro- 



38. 



a second component configured to load re- 
lated data conreiated with the media pro- 
gram for controlling computer system exe- 
cution during presentation of the media 
program; and 

a processor including: 

a third component configured to identity in 
the related data an attribute change to oc- 
cur during presentation of the media pro- 
gram; 

a fourth component configured to identify 
In the related data an index point associat- 
ed with the attribute change for triggering 
the attribute change upon detecting occur- 
rence of the initiating event during presen- 
tation of the media program; 
a fifth component configured to emulate 
presentation of the media program; and 
a sixth component configured to detect the 
Initiating event associated with the index 
point and trigger the attribute change as- 
sociated with the index point. 

Acomputerloadedwith the computer program code 
defined in any one of claims 1 6. 33 or 36. 
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